Hands-free operation of mobile devices may be relevant in a variety of contexts such as in-vehicle operation and disability-related usage scenarios. Initiating mobile device interactivity in a hands-free setting, however, may present a number of challenges. For example, conventional solutions may designate a pre-arranged activation phrase (e.g., “hey computer”) that enables a speech-based user interface for further interaction, wherein audio may be sampled continuously for analysis by a phrase recognizer until the activation phrase is detected. Such an approach may increase power consumption and have a negative impact on battery life.